Peripheral Component Interconnect (PCI) is widely used as a bus standard for connecting computing devices, such as a central processing unit (CPU), hard disk device and graphic controller, and for connecting networking devices, such as a forwarding engine and network controller. PCI Express is standardized as a next-generation PCI standard: According to PCI Express, a parallel bus of PCI is turned into a serial one and communication takes place in a packet method.
FIG. 1 is a block diagram illustrating one example of the configuration of a computer system that uses PCI Express. As shown in FIG. 1, the computer system that uses PCI Express includes a CPU 901, a root complex 902, a memory 903, a PCI Express switch 904, and endpoints 905, 906 and 907.
The CPU 901 performs a central processing process of a computer. The root complex 902 is a root complex of the PCI Express mounted on a host bridge that controls the I/O access from the CPU 901. The memory 903 is connected to the host bridge. The PCI Express switch 904 is connected to the root complex 902. The endpoints 905, 906 and 907 are the endpoints of the PCI Express mounted on computing devices that is connected to the PCI Express switch 904.
As described above, in the computer system that uses PCI Express, there is only one root complex 902 to which the CPU 901 that manages a memory space of the PCI Express can be connected. Therefore, with the above configuration, it is not possible to form a system in which a device is shared by a plurality of computing systems.
FIG. 2 is a block diagram illustrating an example of the internal configuration of the PCI Express switch 904. As shown in FIG. 2, the PCI Express switch 904 includes an upstream PCI-PCI bridge 9041, which is connected in the direction of the root complex; and downstream PCI-PCI bridges 9042, 9043 and 9044, which are connected in the directions of the endpoints. The PCI-PCI bridges 9041, 9042, 9043 and 9044 are connected together through a PCI Express switch internal bus 9045.
For data transferred through the PCI Express system and the PCI Express switch internal bus 9045, a PCI Express frame is used: The PCI Express frame is managed on an address space of the CPU 901. In the downstream PCI-PCI bridges 9042, 9043 and 9044 connected to the upstream PCI-PCI bridge 9041 and the endpoints, only for the PCI Express frame corresponding to an address set by the CPU 901, data is transferred to the root complex 902 or the endpoints 905, 906 and 907.
As described above, in general, the PCI Express system has the structures of a frame transfer method and PCI express switch that are based on the assumption that there is only one CPU 901 on the system. Therefore, in the PCI Express system, with the above configuration, it is not possible to connect a plurality of CPUs on the system.
Therefore, a method of connecting a plurality of CPUs on a PCI Express system has been proposed. For example, what is disclosed in PTL1 is a method of connecting a plurality of CPUs on a system. FIG. 3 is a block diagram illustrating the configuration of a system that can be realized by the method disclosed in PTL1. FIG. 4 is a block diagram illustrating an example of the configuration of a settable PCI Express switch 111 disclosed in PTL1.
The PCI Express switch 904 shown in FIGS. 1 and 2 has one internal bus 9045 and one upstream PCI-PCI bridge 9041. On the other hand, the settable PCI Express switch 911 shown in FIGS. 3 and 4 includes a PCI Express switch internal bus 9116, which corresponds to an upstream PCI-PCI bridge 9111; and a PCI Express switch internal bus 9117, which corresponds to an upstream PCI-PCI bridge 9112. That is, the number of CPUs 901 and 908 being connected equals the number of internal buses 9116 and 9117 prepared, which correspond to the upstream PCI-PCI bridges 9111 and 9112. The connection relationship between the internal buses 9116 and 9117, a downstream PCI-PCI bridge 9113, a downstream PCI-PCI bridge 9114 and a downstream PCI-PCI bridge 9115 is controlled by a bridge controller logic 9118. With the above configuration, it is possible to connect a plurality of CPUs.
For example, what is disclosed in NPL1 is another method of connecting a plurality of CPUs on a system. FIG. 5 is a block diagram illustrating the configuration of a system that can be realized by the method disclosed in NPL1. FIGS. 6 and 7 are block diagrams each illustrating an example of the configuration of a MRA (Multi-Root Aware)-PCI Express switch disclosed in NPL1.
In a system that uses MRA-PCI Express switches, as shown in FIG. 5, it is necessary to put multi-root PCI manager software (MR-PCIM) 952 on any one of CPUs (which is a CPU 901 in the case of FIG. 5) that are connected to the switch. The MR-PCIM 952 manages the configuration state of the switch. It is also possible to connect I/O devices, such as MR endpoints 9505 and 9506, that handle access from a plurality of root complexes 902, 909 and 9021. The setting of the MRA-PCI Express switches is performed from the multi-root PCI manager software 952. Therefore, it is possible for a plurality of the root complexes 902, 909 and 9021 to connect. It is therefore possible for a plurality of the root complexes 902, 909 and 9021 to access devices of the MR endpoints 9505 and 9506. Incidentally, hereinafter, it may be explained that the MR-PCIM 952 executes. However, more specifically, the CPU 901 operates in accordance with the MR-PCIM 952, which is software.
The following describes the configuration of a MRA-PCI Express switch. Incidentally, what is shown in FIG. 5 as an example is the configuration of a system where two MRA-PCI Express switches are used. FIGS. 6 and 7 show the internal configuration of MRA-PCI Express switches 9511 and 9512.
The MRA-PCI Express switch 9511 includes PCI-PCI bridges 95111, 95112, 95122, 95113, 95114 and 95115, which are different from the PCI Express switch 904 shown in FIGS. 1 and 2, in order to accommodate a plurality of the root complexes 902 and 909 and the MR endpoint 9505. The MRA-PCI Express switch 9511 also includes virtual PCI Express switches 95116, 95117, 95118 and 95119, which perform a switching process between bridges; a MRA controller logic unit 95120, which is set and controlled by the MR-PCIM 952; and a setting register 95121, in which the setting information thereof is stored.
Similarly, another MRA-PCI Express switch 9512, which is provided to expand the number of connection ports, includes PCI-PCI bridges 95123, 95124 and 95133, which are different from the PCI Express switch 904 shown in FIGS. 1 and 2. The MRA-PCI Express switch 9512 also includes virtual PCI Express switches 95126, 95127 and 95129, which perform a switching process between bridges; a MRA controller logic unit 95130, which is set and controlled by the MR-PCIM 952; and a setting register 95131, in which the setting information thereof is stored.